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Abstract 

We show that an information-theoretic property of Shannon's entropy 
power, known as concavity of entropy power [7\, can be fruitfuhy employed 
to prove inequahties in sharp form. In particular, the concavity of entropy 
power implies the logarithmic Sobolev inequality, and the Nash's inequality 
with the sharp constant. 

1 Introduction 

In information theory, inequalities constitute a powerful tool to solve communication 
theoretic problems. Due to its wide range of application. Shannon's entropy is at 
the basis of many of these inequalities [S]. Some deeper inequalities were developed 
by Shannon himself in its pioneering 1948 paper |17j. Among other facts. Shannon 
stated the entropy power inequality in order to bound the capacity of non-Gaussian 
additive noise channels. 

In its original version. Shannon's entropy power inequality gives a lower bound 
on Shannon's entropy functional of the sum of independent random variables X, Y 
with densities 

exp (^^H{X + Y)^ > exp + exp , n > 1, (1) 

with equality if X and Y are Gaussian random variables. In inequality ([T]), Shan- 
non's entropy of a random variable X with density is defined as 

H{X) = H{f) = - [ f{v) \ogf{v) dv. (2) 
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Note that Shannon's entropy coincides with Boltzmann's if-functional up to a 
change of sign [H]. The entropy-power 



NiX) = Nif) = exp (3) 

(variance of a Gaussian random variable with the same Shannon's entropy func- 
tional) is maximum and equal to the variance when the random variable is Gaussian, 
and thus, the essence of ([1]) is that the sum of independent random variables tends 
to be more Gaussian than one or both of the individual components. 

An interesting property of the entropy power has been discovered in 1985 by 
Costa [7j. Let f{v,t) denote the solution to the Cauchy problem for the heat equa- 
tion 

%^ = A/(,,0. (4) 

posed in the whole space j9], corresponding to the initial value /(f), which we 
assume to be a probability density function. Note that for t > 0, the solution to the 
heat equation (jl]) can be written as f{v,t) = f * M2t{v), where as usual * denotes 
convolution, and Mt{v) is the Gaussian density in M" of variance nt 



1 f\v 



|2 



Mt(v) = - --rrexp — . (5) 

Costa [7] proved that, for any given probability density function / different from 
the Gaussian density, N{f * M2t) is a concave function of time 

^N{f * M^t) < 0. (6) 

The concavity property of entropy power outlines a new property of Gaussian func- 
tions. Indeed, the entropy power of a Gaussian function coincides with its variance, 
so that the entropy power of the fundamental solution to the heat equation is a 
linear function of time. This linearity is restricted to Gaussian densities. 

Later, the original proof of Costa [7] has been simplified in [TOj ITT] , by an ar- 
gument based on the Blachman-Stam inequality More recently, a short and 
physically relevant proof has been obtained by Villani [22], resorting to some old 
ideas of McKean [15j. The proof of Villani establishes a deep link between the con- 
cavity of entropy power and the logarithmic Sobolev inequality. It is remarkable 
that the same ideas of McKean have been seminal for a new proof of logarithmic 
Sobolev inequality published some years ago [19j. 

The concavity of entropy power involves the solution to the heat equation. This 
basic fact includes the concavity of entropy power in the set of inequalities which, 
in alternative to other ways of proof, can be derived by means of the heat equation. 
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Indeed, the linear diffusion equation [2(T] represents a powerful instrument to obtain 
a number of mathematical inequalities in sharp form. 

This maybe not so well-known property goes back more or less to half a century 
ago, when independently from each others, researchers from information theory [ 18| 
|3], kinetic theory [15], and probability theory [14j, established a useful connection 
between Boltzmann's if-functional and Fisher information exactly by means of the 
solution to the heat equation. 

In this note, we proceed along the same lines to show that the concavity of 
entropy power (a property of the solution to the heat equation) allows to prove as 
corollaries important inequalities, like the logarithmic Sobolev inequality and Nash's 
inequality in sharp form. 

Connections between the logarithmic Sobolev inequality and Nash's inequality 
in sharp form are well known. Beckner [2] used the former to prove the latter 
inequality with a sharp constant, thus obtaining Nash's inequality from an argument 
different from the argument used by Carlen and Loss [1]. The best constant for 
Nash's inequality was indeed calculated by Carlen and Loss, who observed that this 
inequality is equivalent to the Poincare inequality in a suitable ball of M". 

The next Section is devoted to the proof of the concavity of entropy power. We 
will be mainly concerned with the key ideas behind this proof, as well as to the 
analogies between this proof and analogous ones based on the solution to the heat 
equation. Section [3] will be devoted to show that the logarithmic Sobolev inequality 
is a direct consequence of the concavity of entropy power, which in some cases allows 
to prove the previous inequality with a remainder. 

Last, in Section H] we will show how Nash's inequality with a sharp constant 
follows from the concavity of entropy power. The proof is very simple, and makes use 
only of elementary inequalities, as well as of well-known properties of the logarithmic 
function. 

2 The concavity of entropy power 

The proof of concavity requires to evaluate two time derivatives of the entropy power, 
along the solution f{v,t) to the heat equation. The first derivative of the entropy 
power is easily evaluated resorting to the so-called DeBruijn's identity 



which connects Shannon's entropy functional with the Fisher information of a ran- 
dom variable with density 



Hfit)) 



dt 



H{f{t)), t>0, 



(7) 
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Using identity ([7]) we get 



Hence 



exp { jH(f[t)) \ I(f[t)) 



Let us set 

T/(t) = [fit)) = exp {^^(/W)} n/W). (9) 

Then, the concavity of entropy power can be rephrased as the decreasing in time 
property of the functional T f{t) along the solution to the heat equation. If 

_ . (10) 

denotes the derivative of Fisher information along the solution to the heat equation, 
we obtain 

|x,,.exp{l.,;„,}(«.f.a,„ 

exp\-H{m)] (-j{m) + -i{my 

Hence, ^ f{t) is non increasing if and only if 

> -iu{t)f. (11) 

n 

It is interesting to remark that, aiming in proving the old conjecture that subsequent 
derivatives of Boltzmann's if-functional alternate in sign, the functional J(/(t)) was 
first considered by McKean [TB]. In one dimension, inequality (ITT]) is essentially due 
to him. Let us repeat his highlighting idea. In the one dimensional case one has 



/(f) 

while 

McKean observed that J{f) is positive. In fact, resorting to integration by parts, 
J(/) can be rewritten as 
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Having this formula in mind, consider that, for any constant A > 
< 2 



fiv) f{vf 

f'ivr 



J{f) + 2X' + 4A ^ (^nv) - ^) dv = J{f) + 2X' - 4XI{f). 

Choosing A = /(/) shows (fTT]) for n = 1. 

Note that equahty in (fTT]) holds if and only if / is a Gaussian density. In fact, 
the condition 

fiv) f\vf 



+ A = 0, 



can be rewritten as 



which corresponds to 



dv 



log/(t;) 



-Xv'^ + bv + c. 



(14) 



Joining condition (IT^ with the fact that f{v) has to be a probability density, we 
conclude. 

The argument of McKean was used by Villani j22] to obtain ( ITTj) for n > 1. In 
the general n-dimensional situation, Villani proved the formula 



n „ 

^(/) = 2E / 



dvidvj 



log/ 



fdv 



^dldl 

f dvidvj P dvi dvj 

By means of (fT5|) . the nonnegative quantity 

'1 _ 1 df df 



fdv. 



(15) 



n „ 

m = E / 



H 2 



/ dvidvj p dvi dvj ~'~ '^^^'^ 



with the choice A = I{f)/n, allows to recover inequality (fTTj) for n > 1. This proves 
the concavity property of entropy power. 

To show that the concavity of entropy power has significant consequences, we 
need to outline a further property of the functional T f{t) [20j. Given a function 
g{v) > 0, f G M", let us consider the scaling (dilation) 



g{v) ^ ga{v) = a''g{av) , a > 0, 



(16) 



5 



which preserves the total mass of the function g. By direct inspection, it is immediate 
to conclude that Shannon's entropy is such that, if the probability density is 
defined as in (fT6|l 

H{fa) = H{f)-nloga. (17) 
Since Fisher's information scales according to 

V fa{v) V f{v) 



one concludes that the functional T f{t) is invariant with respect to the scaling (flGil 
of the solution f{v,t) of the heat equation. Therefore, for any constant a> 

Tfifit)) = Tf iUt)) . (19) 

Property (fT9|) allows to identify the long-time behavior of the functional T f{t). Un- 
less the initial value f{v) in the heat equation is a Gaussian function, the functional 
T(t) is monotone decreasing, and it will reach its eventual minimum value as time 
t — )■ oo. The computation of the limit value uses in a substantial way the scaling 
invariance property. In fact, at each time t > 0, the value of T/(t) does not change 
if we scale the argument f{v,t) according to 

f{v,t)^F{v,t)= (^VT+2ty f{vVT+2t,t), (20) 

which is such that the initial value f{v) is left unchanged. On the other hand, it is 
well-known that (cfr. for example [^) 

lim F{v,t) = Mi{v) (21) 



t—^oo 



where, according to ([5]) Mi{v) is the Gaussian density in M" of variance equal to 
n. Likewise, the limit value of Tf{t) does not change if we scale the limit Gaussian 
function according to fll6p in order to have a variance different from one. Therefore, 
passing to the limit one obtains, for any a > 0, the inequality 

exp |^^(/)} /(/) > exp |^^(M.)| /(M^). (22) 

3 The logarithmic Sobolev inequahty 

Let us assume that f{v) in fl2^ is a probability density function, so that = 1. 

In this case, for any given probability density f{v), and any a > inequality (l22il 
takes the form 

^^^^ >expl--{H{f)-H{M^))\. (23) 



6 



Since 

/(M.) = -, 
cr 

while 

H{M,) = -\og27ia + -, 
using that > 1 — x we obtain from ( 123|) 

fiv) log f{v)dv + n+'^ log 27ra < ^ / ^^^^ d^. (24) 

Inequality fl24p is nothing but the logarithmic Sobolev inequality by Gross [12j, 
written in an equivalent form. 

Consider now the case in which the probability density f{v) of the random 
variable X is such that the second moment of X is bounded. Then, for any a such 
that 

a > - / 

it holds 



a>- \v\y{v)dv, (25) 



-H{f) + H{M,)= [ f{v)\ogf{v)dv- I M,{v)\ogM,{v)dv = 
I ^^^)l°g/i7T\^^ + ^ / \v\HM^-f{v))dv>[ f{v)\og^dv. 



By the Csiszar-KuUback inequality 



2 / f{v)\og^^^dv>\\f-MA\l.. (26) 



By expanding the right-hand side of inequality (123|) up to the second order, we end 
up with the inequality 



The right-hand side of fl27p constitutes an improvement of the logarithmic Sobolev 
inequality, in that, at least when the density function involved into inequality (l23i) 
has bounded second moment, and it is different from a Gaussian density, it is possible 
to quantify the positivity of the difference between the right and left sides of ( !23|) 
in terms of the distance of it from the manifold of the Gaussian densities, with a 
precise estimate of this distance in terms of the L^-norm. 
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4 Nash's inequality revisited 

A second interesting consequence of the concavity of entropy power is the proof of 
a reinforced version of Nash's inequahty [TB]. To this aim, note that the right-hand 
side of inequahty (122|) . thanks to the scahng invariance property of Tj, does not 
depend of a. Since for any given cr > 

11 
a 



while 



The choice 



gives 
and 



H{M„) = -\og27ra+-. 



a = a = {27ce)-\ (28) 
I{M^) = 27Ten, 



H{M^) = 0. 

Thus, substituting the value a = o" in (12^ we obtain the inequality 

exp|^/7(/)| /(/) >27ren. (29) 

Inequality fl29l) is know under the name of Isoperimetric Inequality for Entropies 
(cfr. [llj for a different proof). 

The case in which /(f) > is a density of mass different from 1, leads to a 
modified inequality. Let us set 

^ = / f(v) dv. 

Then, the function (j){v) = f{v)/fi is a probability density, which satisfies (129|) . 
Therefore 

/(/i0) = fil{4>) > /i J(M^)exp 1-H{mA exp ( --//(</>) 



J I n 

fiI{M„) exp <; -{H{M,) - log/i)l exp 1--{H{4>) - \ogfi) 
n J I ^ 

/i J(M^) exp <; --HUM^ I exp I ---HU(p) I . (30) 
n n J [ n /i J 
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In ( 150]) we used the identity 

H{^(f)) = fiH{(f)) — /i log fi. 
Setting now a = (x, as given by (125]) . we conclude with the inequality 

/(/) > 27Ten ll/IUi exp [H{f) - Wfh^ log WfhA } , (31) 



which clearly holds for any integrable function f[v) > 0. 

Given a probability density function g{v), let us set f{v) = g^{v). In this case 

H{f)=H{g^) = - [ g\v)\ogg\v)dv = -2 f {g{v)\ogg{v)) g{v) dv. 

Since the function h{r) = rlogr is convex, and \\g\\L^ = 1, Jensen's inequality 
implies 

-Hig^)>2[ g\v)dv\ogf g\v)dv. (32) 
Using (152]) into ( 15T]) gives 

J(/)>27ren/" /(f ) rff ■ exp |- log /" g^{v)dv\=([ g^iv)dv] . (33) 
Using the identity 

I{g') = A[ \Vg{v)\'dv 



we obtain from (??) the classical Nash's inequality in sharp form 

l+2/n 2 



g'{v)dv\ < / \Vgiv)\'dv (34) 

Inequality (l34l) clearly holds for all probability density functions g{v). Note that, if 
llfi'lUi 7^ Ij (IM]) implies 

/ /• \ i+2/n / r \^/nr 
/ ^?'(t;)rft; < / |^?(t;)rft; / |V(7(t;)p rft;. (35) 

The constant 2 /{-Ken) in (l35]) is sharp. 
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5 Conclusions 



The concavity of entropy power is a property of Shannon's entropy which has un- 
expected consequences in terms of functional inequahties. In this paper we made 
exphcit the hnks between this property and the logarithmic Sobolev inequality by 
Gross [12], as well as Nash's inequality |16) . In both cases, the concavity of entropy 
power allows to improve these inequalities. In the case of the logarithmic Sobolev 
inequality, it is shown that, for densities with bounded second moment, it is possible 
to give a precise estimate of the distance between the density and the manifold of 
Gaussian functions, which are known to saturate the inequality. Also, the clearness 
of the physical idea, and the relative simplicity of the underlying computations, are 
in favor of the information-theoretic proof of these inequalities. 
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